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Abstract 

Evaluation is primarily a measure of the effects and elaboration on the current performance of a program. It facili- 
tates identification and choice of the ways to achieve the intended program objectives. The evaluation includes selec- 
tion of samples, as well as collection and analysis of data. 

This paper discusses the issue of the allocation of samples for the population for which programs have been 
implemented in a phased manner. This needs special attention due to the temporal impact of allocations on the suc- 
cessive units. A method of proportional allocation has been proposed for the estimation of the sample size in different 
phases under the assumption that the impact of the latter phase is a multiplicative product of constant factor and the 
former phase. 
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Introduction 

Evaluation is the process of determining the present 
existing state of units under the influence of an imple- 
mented program or developmental activities. It deter- 
mines the positive and negative impacts and associated 
losses and gains that directly result from the program. 
The results can be used as policy instruments for future 
planning and can also provide some tools for retrospec- 
tively determining the performance of individuals for 
implementing programs. 

Generally the developmental program such as pov- 
erty alleviation, literacy drive, health scheme etc. is 



being implemented on units across a large geographi- 
cal area. Therefore, evaluation of all units is generally 
not feasible, thus sample evaluation is recommended. 
Therefore, all the issues pertaining to sampling should 
be taken into consideration for precise evaluation. It 
includes nature of population units, efficient sample 
size, and estimation procedures depending on the ap- 
plied sampling scheme. The program which has been 
implemented under different phases leads to the time- 
dependent impact, therefore for evaluation; temporal 
impacts cannot be ignored. Therefore, for estimating 
the parameter of interest, the sample size and its alloca- 
tion should consider the temporal effects of the popula- 
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tion units. Sampling theory provides a solution for the 
allocation of sample size (Cochran 1977). 

The impact distribution of such populations is 
usually highly skewed with long tails in one direction 
due to time dependent impacts. This makes popula- 
tion units heterogeneous and stratified sampling may 
be suitable. The different phases may be considered 
strata, as program implementation attracted all units of 
a phase at a time. Therefore, the essence of stratified 
sampling designs i.e. the number of the strata formed; 
and the construction of the stratum boundaries will be 
addressed automatically. However, sample allocation 
within a strata is tactical keeping in view the time-de- 
pendent impact on the population units. It means, the 
initial units may have different impacts and, hence, the 
response than the latter stage units. Therefore, for uni- 
formity, a weight may be assigned depending on the re- 
alised impacts on units. This will nullify the impacts of 
the implementation phase and minimise the bias on the 
resulting estimates of impact evaluation. 

The large sample will yield for waste of recourse, 
and the small sample may provide no efficient result. 
Therefore to get a precise estimate of the parameter 
for impact evaluation, the impact has to be account for 
through assigning temporal weights to these units, as 
per the actual time or phase of implementation of the 
program for those units. Lacking these accounting will 
lead to improper estimation of impact evaluation. This 
assigning of weight is possible, if the influence pattern 
of the impact is known. The lack of this information 
affects the rationale of practical decisions that will be 
derived from the estimation. 

The issue of finding the stratum boundaries to 
minimize the variance of the stratified sample estima- 
tor was discussed by several researchers based on the 
population characteristics (Dalenius 1950; Dalenius and 
Hodges 1957; Lavallee and Hidiroglou 1988; Horgan 
2006). This heterogeneity in the population may be in- 
trinsic in nature or may be arisen due to some artificial 
or extrinsic causes, e.g. the varied response in popula- 
tion units due to the operation of the developmental pro- 
gram in a phased manner. The variation in response to 
a specified unit is related to the implementation phase 
of the program. That is, the units, which fall in the first 
phase, will be having more time to realize the actual 
effect of the program, therefore will have a different 
impact, logically. This kind of problem of sample al- 



location in different strata for impact evaluation of any 
program/projects implemented in a phased manner was 
addressed for the additive impacts in respective units by 
Pandey and Verma 2008. 

For efficient estimation of impact evaluation in 
stratified sampling, the pertinent point is allocation of 
the sample size in different strata. Conventionally, it 
can be decided by considering three factors, viz; total 
number of units in the stratum, variability within the 
stratum, and cost of taking observations per sampling 
unit in the stratum. However, it becomes more critical, 
as the phased rational operation influences the study 
variable. The conventional allocation of the sample size 
by assigning weight in different strata depending on the 
characteristics of strata will not account for this vari- 
ability. Therefore, it is logical to consider the time-de- 
pendent response for weight, so that the unit’s response 
should be free from phase implementation. This paper 
attempts to address the issue of allocation of samples in 
different phases through the integration of this informa- 
tion for weight estimation with the assumption that the 
response in successive phases is governed by the multi- 
plicative effect. 

Sample size determination in different 

STRATA 

The sample size in different strata is a priori fixed sci- 
entifically. The allocation of the sample to different 
strata is based on the principle of optimum allocation 
(Sukhatme et. al. 1984). The methods for sample alloca- 
tion in different strata are equal allocation, proportional 
allocation and Neyman allocation (Cochran 1977). 

Symbolically, let there are N units/villages/sectors, 
where the program has been implemented in h (say) 
phases. The phase is generally based on an administra- 
tive regions, therefore may be treated as a stratification 
criterion. Then 

N = f j N i ( 1 ) 

/=1 

A total of ‘n’ sample will be selected randomly with 
pre-decided sampling scheme for impact evaluation. 
The sample size in different strata will be 

« = ( 2 ) 

/= 1 
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The proportional allocation is considered for deter- 
mination of the sample size. It will be 

N. 

n. = n — (3) 
' N 

Here, n is not true for those populations which 
attract some external impact with a lagged response. 
These allocations used available information about the 
population with consideration that the study variable is 
without influence of external factors. Therefore, these 
methods are not suitable for those situations where the 
study variables are under influence due to some exter- 
nal interventions such as the present case, in which the 
impacts are time-dependent. 

Therefore, theoretically, if this is used, the impact 
evaluation will be erroneous. This may also happen 
due to the fact that the realized impact under different 
phases will be different due to the time lag in imple- 
mentation (Pandey and Verma, 2008). It means that, be- 
cause of the dynamic nature of the population under the 
influence of temporal impacts from phase to phase, the 
general weight W. = N. / N cannot be applied for the 
estimation of sample mean, sample variance and alloca- 
tion of sample size. 

This can be addressed by selecting units in such 
a way that the impact on all units should be treated iden- 
tically by applying some artificial measure. This can be 
achieved through assigning weight to each stratum de- 
pending on the impact of the program being implement- 
ed. These weights can be used to obtain the hypothetical 
population stratum size by making all the units of the 
population similar for evaluation. 

To achieve the weight, it may be considered that the 
impact of the program is uniformly distributed within 
each phases and that it is constant and multiplicative in 
nature with respect to different phases. In other words, 
if there are h phases, and if the impact of the last phase 
(the most recent one) is M, and the impact of the im- 
mediately preceding phase is Q then Q = MU, where 
U is the constant multiplicative factor i.e. improvement 
factor due to the time lag (realized impact) between the 
preceding and succeeding ones. In other words, we can 
say that the units, which received the programs in the 
preceding to last phases, will have U times more impact 
than the units, which receive the last phase of the pro- 
gram with the above assumptions. This multiplicative 
factor may assume values greater than one (U > 1), with 



the assumptions that the developmental program will be 
implemented with the objectives of improvement over 
time. 

Mathematically, let us consider a population of size 
N containing the population units N v N 2 , ..., N h for the 
implementation of the h phase of the developmental pro- 
gram. Suppose that the development program is imple- 
mented in the h phase with the impact of the last phase 
being M, then the impact of the immediately preceding 
one will be MU, where U is defined above. Therefore, 
for the first phase, if there are N x units, then the total 
actual impact will be MU (/? - 1) times more than the 
last beneficiaries. Hence, for comparison with the last 
units, the hypothetical stratum size may be (. MU (h ~' ) )N v 
In the second phase, the development program consid- 
ers N, units. Hence, the total hypothetical population for 
that phase will be ( MU°'~ 2) )N r In general, there will 
be {MU (h ~ l) )N beneficiaries for the i-th stratum. There 
will be (. M)N . units - beneficiaries in the last phase of 
implementation of the development program. For more 
clarity, this has been presented in Tab. 1. A similar prob- 
lem with additive impacts of the programme in differ- 
ent phases was considered by Pandey and Verma, 2008. 
The weight for such impacts was defined as ( h — i + 1) 
N., where h is the additive impact factor between two 
successive phases. 

Based on the above logic, the weight will be based 
on the impact and implementation phase. Therefore, the 
general formula for the weight is: 

(MU {h ~ n )N. 

w. = (4) 

' N„ 

where: 

h - number of the development program phase, 
i - stratum number, i= 1,2,3, . . ., h, 

N. - actual numbers of beneficiaries in the /-th stra- 
tum, 

N h - total numbers of hypothetical units in the popula- 
tion adjusted by the impact or phase factor with 
the product of the actual population i.e. the sum 
of the units adjusted by the impact factor, 

U - constant multiplicative factor of the preceding 
phase with the succeeding one, 

M - realised impact of the last phase. 
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Estimation of the mean and variance 

FOR THE PROPOSED METHOD 



The population mean for a program implemented in h 
phases i.e. h strata will be 

i= 1 

for the study variable K where Y. is the mean of the z'-th 
phase. 

The unbiased estimator of the population mean Y is: 

y«=im (5) 

i=l 



where y is the unbiased estimate of Y for the z-tli stra- 
tum or phase. The variance of y is given as follows 
(Sukhatme, et ah, 1984): 



nyj=f 

i = 1 




( 6 ) 



The formula shows that the variance of y depends 
only on the variances of the estimates S 1 of the indi- 
vidual stratum means Y with TV. number of units and the 

/ r 

sample size of n.. 

However, for estimating the mean of the studied 
variable for the development program implemented in 
different phases, one cannot use the conventional weight 
W for the stratum due to non accounting of the tempo- 
ral impact. Therefore, the modified weight co defined in 
equation (4) may be more realistic at least theoretically. 
Therefore, on a logical basis, the unbiased estimator of 
the population mean is: 

h 

y slm = Y w ,y i ( 7 ) 

/= l 



where y is the mean of the character under stratified 

y stm 

sampling with modified weights. 

Similarly, the variance of y will be estimated as 



*w 2 S 2 * w.5 2 

V(y )=Y-^-Y- l ~ l - 
s,m t? n " TV. 



( 8 ) 



New Proportional Sample Allocation 
Method 

This section addresses the allocation of samples to 
different strata for the present case. The conventional 
proportional allocation will not be applied for allo- 



cating samples to strata for impact evaluation of the 
development program due to non-accounting of the 
temporal variation. Hence the modified method is pro- 
posed in the paper. In this allocation, the number of 
units selected from each stratum depends directly on 
the number of units in the stratum and the temporal 
effect of the developmental program. Mathematically, 
it can be derived as: 

- from the z-tli strata of the stratified population 

n. °cT V. (9) 

- weight assigning due to program effect 

n. °c (M/ ( *~°) (10) 

- based on (9) and (10) we get 

«. °c(M/ (,M) )TV. (11) 

- the equation can be rewritten as with constant k. 

n. = k(MU^ h ~ n )N. (12) 

- taking the summation on both sides in (12) will re- 
sult into the total sample size, 

Yn=n=kY J {MU {h - i) )N i (13) 

/= l ;=i 



however, for proportional allocation with a hypo- 
thetical population 



n = kN„ 



(14) 



now putting the value of k in equation (12), the sam- 
ple size from the z-th strata will be 



MU ik ~°N. 



TV 



Hy 



(15) 



- since w. = 

i 



MU {h - l) N. 

I 



N 



Hy 



hence we can write that 



n. = CO.n (16) 

This equation (16) can be used to allocate sample 
sizes in different strata for impact evaluation of a de- 
velopment program, which has been implemented in 
phases and have a multiplicative impact in respective 
units. 
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Empirical Study for Sample Allocation 

Let us consider that the development program is being 
implemented in five phases in different villages. In the 
first phase the development program is started in 20 
villages. In the second phase, it is implemented in an- 
other 30 villages. Similarly, the development program 
is being implemented in different 40, 50 and 60 vil- 
lages in the third, fourth and fifth phases, respectively. 
It was assumed that the development program had mul- 



tiplicative effects in succeeding villages. In this case, 
conventional allocation may lead to improper selection, 
keeping in view the high impact in initial villages. The 
allocation of samples in different phases using conven- 
tional proportional allocation and new proportional al- 
location is illustrated in Tab. 2 for comparative analy- 
sis. As seen in the Table, there are differences in the 
allocation of the sample size in different phases with 
more representation of the initial phases in the pro- 
posed method. 



Tab. 1. Formula for the allocation of the sample size for impact evaluation 



Stratum 

number 


Stratum 

population 

size 


Stratum 
sample size, 
say 


Impact 
of program 
(Impact Factor) 


Stratum population size 
due to calibration 
of impact 

(Hypothetical Size) 


Estimated Stratum 
Weight with calibration 
of impact 


Stratum 

sample 

size 


1 


K 


", 


MUW 


(. MU^)N l 


(. MU ih -' ) )N l 

N ~ C ° V 

Hy 

Say 


nco { 


2 


n 2 


n i 


MU 


(MU ( "- 2) )N 2 


( MU^)N . 

= f0 2’ 

% 

Say 


nco 2 


3 




»3 


MU^ 


(MU^)N 3 


( MU«~V)N , 

- = (o v 

% 

Say 


nco 3 


4 




»4 


MU 


(MUM)N t 


( ~MU^)N , 

N =C °* 

Hy 

Say 


nco A 
















h — 1 




”k-l 


MU 


MUN hl 


MWV, 

N 

Hy 

Say 


nco „-i 


h 




n h 


M 


MN, 


MN. 

N h 

Hy 


nco, 

h 


Total 


N 


n 


M( 1 - U") 
1 -M 


N Hy (Say) 


CO 1 + C0 2 + ... + CD n = 1 


n 
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Tab. 2. Numerical example for a five-year implementation program 



Stratum 

number 


Population 

size 

N 


Stratum sample 
size (proportional 
allocation, 
n = 20) 


Impact factor 
(M= 2, U= 2) 


Hypothetical 

stratum 

population 


Weight 

CO. 


Sample size for program 
evaluation with 
Proposed Methods 
n 


1 


20 


2 


32 


640 


640/1760 = 0.36 


7.20 = 7 


2 


30 


3 


16 


480 


480/1760 = 0.27 


5.40 = 6 


3 


40 


4 


8 


320 


320/1760 = 0.18 


3.60 = 4 


4 


50 


5 


4 


200 


200/1760 = 0.11 


2.20 = 2 


5 


60 


6 


2 


120 


120/1760 = 0.07 


1.40= 1 


Total 


200 


20 


62 


1760 


1 


20 



Conclusions 

Mid-term impact evaluation of any program/projects 
is presently in vogue in the developing countries. 
The allocation of samples in different strata based on 
phased implementations needs special attention due to 
its temporal impact on the units. The accounting of 
temporal impact for allocation of samples may provide 
a better mid-term evaluation of the program at least 
based on the theoretical logic than the conventional 
approach i.e. inclusion of impacts. Pandey and Verma, 
2008 have reported the case of sample allocation for 
the additive impacts in successive strata. This study 
considers the issue of implementation of the develop- 
mental program in phased manner with multiplicative 
effects and reported an empirical study of sample al- 
location only. This also gives different estimates of the 
sample size for each stratum i.e. by considering the 
impact of each phase. 

Overall, it appears that: compared to the classical 
sampling analysis for the pre-specified margin of error 
approach, as well as the conventional approach, the pro- 
posed method may result in a better and theoretically 
efficient estimate keeping in view the real coverage in- 
corporating the significance of impacts. The theoretical 
efficiency may seem to be logical due to the incorpora- 



tion of the influential components in the form of weight 

for sample allocation in different strata. 
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